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ABSTRACT 

A  procedure  for  determining  the  rank  of  a  quadratic  form 
is  outlined  by  Cramer  [1]  and  Hald  [2],   Additional  theoret- 
ical verification  of  this  procedure  is  presented  and  the 
results  are  illustrated  with  applications  in  the  analysis 
of  variance. 
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I.   INTRODUCTION 

A  comparison  of  several  sets  of  observations  drawn  from 
normally  distributed  populations  can  be  performed  by  means 
of  the  statistical  procedure  known  as  the  analysis  of  var- 
iance.  The  justification  for  the  statistical  procedure  in 
the  analysis  of  variance  depends  directly  uoon  the  applica- 
tion of  Cochran's  Theorem  stated  below. 

Cochran's  Theorem  Let  X  =  (X.  ,X~  , .  .  .  ,X  )  be  distributed 
as  N  (0,1)  and  suppose 


n    ~     k 

I     x2,=  I     Q.(X) 

1=1      1=1 

where  Q.  is  a  quadratic  form  of  rank  n. ,  i=l,...,k.   Then 
Q, (X) ,  ...,Q,  (X)  are  mutually  independent  and  Q. (X)  is  dis- 
tributed with  n.  degrees  of  freedom,  i=l,...,k,  if  and  only 
if 

k 

T   n.  =  n. 
i=l  X 

Cochran's  Theorem  formally  relates  (a)  the  degrees  of 

2 

freedom  of  a  x   random  variable,  Q(X) ,  with  the  rank  of  its 

associated  observed  variate,  0(x) ,  when  X  is  N  (0,1)  and 

2 

(b)  the  sum  of  the  degrees  of  freedom  of  a  set  of  x   random 

variables  with  the  dimension  of  the  random  vector  X;  the 

2 
independence  among  the  x   random  variables  is  a  consequence 


of  this  relationship  [2], 


Hence,  if  a  statistician  is  collecting  a  random  sample 
from  a  normally  distributed  population  and  if  the  sum  of 
squares  of  the  observations  from  the  random  sample,  x  = 
(x, , ... ,x  ) ,  equals  the  sum  of  several  quadratic  forms  in 
x,  say  Q-,,...,QK,  he  must  show  that  the  sum  of  the  ranks  of 
these  quadratic  forms  is  equal  to  the  number  of  total  obser- 
vations before  he  can  conclude  independence  of  the  Q. 's, 

2 
i=l,...,K,  and  assign  x   probability  distributions  to  them. 

In  the  analysis  of  variance  these  quadratic  forms,  when 

divided  by  their  ranks,  represent  independent  estimates  of 

2 

an  unknown  variance  a   associated  with  a  random  variable 

2 
vector  Y  which  is  N  (y,a  I).   The  N  (0,1)  hypothesis  of 

Cochran's  Theorem  becomes  satisfied  upon  letting 

2 
Since  the  normalized  quotient  of  independent  x   variables 

is  an  F-random  variable,  F-statistics ,  which  are  used  to 

test  hypothesis  concerning  y,  can  be  formed  from  the  ratio 

of  the  quadratic  forms  in  x. 

In  turn  then,  it  can  be  seen  that  the  application  of 

Cochran's  Theorem  depends  upon  the  determination  of  the  ranks 

of  the  quadratic  forms  in  x  in  the  relation 


n    2 

I      xi  =  Q1(x)  +  Q2(x)  +  .  .  .  +  QR(x)  . 
i=l 

Cramer  [1]  gives  a  procedure  for  determining  the  rank 


of  quadratic  forms  as  follows 


2  2 

"....   If  Q  may  be  written  in  the  form  Q  =  L,  +  ...  +  L, 

where  the  L.  are  linear  functions  of  x. ,...,x   and  if  there 
1  In 

are  exactly  m  independent  linear  relations   between  the  L. , 

then  the  rank  of  Q  is  k-m " 

Cramer  calls  this  a  proposition. 

Likewise,  Hald  [2]  states  a  similar  procedure  as  a  def- 
inition. 

"The  number  of  degrees  of  freedom  for  a  set  of  variables, 
L, , . . .  ,L,  ,  will  be  defined  as  follows:   Let  the  k  variables 
L,  , . . .  ,L,  be  linear  functions  of  n  stochastically  independ- 
ent variables,  x, , . . . ,x  ,  which  are  assumed  to  be  normallv 
distributed  with  parameters  (0,1).   If  m  independent  lin- 
ear relations   exist  between  the  k  variables,  L.  , .  . .  ,L,  , 
the  number  of  degrees  of  freedom  is  k-m.   The  number  of 
degrees  of  freedom  for  the  sum  of  squares 

k 
Q  =  I      L2 
i=l   X 

is  defined  as  the  number  of  degrees  of  freedom  for  the  k 

variables  L.  , . . .  ,L.  . " 
1      k 


Both  Cramer  and  Hald  point  out  that  the  m  linear  rela- 
tions between  the  L's  are  iinearly  independent.   Formally, 
a  linear  relation  has  the  form 

c,  L,  +  .  .  .  +  c,  L,  =0 
11  k  k 

where  the  c^  are  constants  and  not  all  zero.   When  several 
linear  relations  of  this  form  exist,  they  are  called  inde- 
pendent, if  the  corresponding  vectors  c  =  (c,  ,...,c,)  are 
linearly  independent. 


Cramer's  proposition  is  stated,  but  not  proved.   Hald,  on 
the  other  hand,  precludes  the  necessity  of  proving  the  same 
proposition  by  calling  it  a  definition.   In  either  exposition 
no  detailed  theoretical  verification  is  made  of  the  procedure 
for  determining  the  rank  of  quadratic  forms.   It  is  the  in- 
tent of  this  thesis  to  (1)  state  and  prove  basic  theorems 
which  can  be  used  to  determine  the  rank  of  quadratic  forms 
in  the  way  presented  by  Cramer  and  Hald  and  (2)  illustrate 
the  use  of  these  theorems  in  the  analysis  of  variance. 


II.   MATHEMATICAL  BACKGROUND 

The  theory  which  is  to  be  developed  concerning  the  rank 
of  quadratic  forms  depends  upon  several  mathematical  results 
from  matrix  algebra  [3,4].   This  chapter  outlines  the  perti- 
nent mathematical  definitions  and  theorems  that  are  necessary 
to  develop  this  theory. 

A.   MATRICES 

A  matrix  A  has  elements  denoted  by  a. .  where  i  refers  to 
the  row  and  j  to  the  column.   If  A  denotes  the  matrix,  then 
A1  denotes  the  transpose  of  A,  and  A   ,  the  inverse  of  A. 
The  symbol  |a|  is  used  to  denote  the  determinant  of  A.   The 
identity  matrix  is  denoted  by  I;  and  0,  the  null  matrix. 
The  dimension  of  a  matrix  is  the  number  of  rows  by  the  num- 
ber of  its  columns,  e.g.,  nxm.   A  matrix  A  of  dimension  nxl 

2 
is  called  a  column  vector  ;  its  transpose  A' ,  a  row  vector. 

The  rank  of  a  matrix  A  is  denoted  by  r(A) .   Euclidian  n-space 

is  symbolized  by  E  . 

Given  matrices  A  =  (a. .)  and  B  =  (b. .)  where  the  number 

-ID  ID 

of  columns  of  A  equals  the  number  of  rows  of  B,  the  product 

AB  =  C  =  (c. ,)  is  defined  as  the  matrix  C  with  the  oq 
ID  - 


2 

Column  vectors  will  be  indicated  by  round  brackets,  as 

x  =  (x,,...,x  );  row  vectors  will  be  indicated  by  square 
brackets,  as  x  =  [x,  ,...,x  ]a 


n 
element  equal  to 

k 


a  ,  b,  .   A  +  B  =  C  gives  a .  .  +  b .  .  =  c .  . 
=!  Pk  kq  ID    ID    iD 


provided  A  and  B  have  the  same  dimension.  If  k  is  a  scalar 
and  A  a  matrix,  then  kA  means  the  matrix  whose  ij  element 
is  ka. ..  A  diagonal  matrix  D  is  a  square  matrix  whose  off- 
diagonal  elements  are  all  zero;  D  =  (d. .)  where  d. .  =  0  if 

iD         ID 

i  /  j,  A  matrix  is  called  symmetric  whenever  A  =  A1 .  If  C 
is  an  nxn  matrix  such  that  CC '  =  I,  then  C  is  said  to  be  an 
orthogonal  matrix,  and  C'  =  C 

Theorem  2.1   r(AA')  =  r(A)  =  r(A'). 

3 
Theorem  2.2   Let  A  be  nxn,  symmetric  and  non-negative. 

Then  there  exists  a  non-singular  matrix  C  such  that  C'AC  = 

(d. 6 . . )  where  d. {0 ,1} ,  i=l,...,n  and  the  rank  of  A  equals 

the'  number  of  non-zero  d.  's. 

i 

Theorem  2 . 3   If  A  is  an  mxn  matrix  of  rank  r,  and  if  B 
is  an  nxq  matrix  such  that  AB  =  0 ,  then  the  rank  of  B  cannot 
exceed  n-r. 

Theorem  2 . 4   Consider  the  sum  of  k  matrices  of  the  same 

dimension,  A,  +  A«  +  .  .  •  +  A,  ,  then 

k         k 
r(  [  A,)<   J   r(A  )  . 
1=1   1    1=1    x 


B.   QUADRATIC  FORMS 

If  A  is  an  nxn  matrix  and  x  =  (x, ,...,x  )  is  an  nxl  vec- 

-      1 '      n 

+■  v» 
tor  with  i    element  x.  then 

i 


3 

The  symbol  6 . .  is  called  the  Kronecker  delta  and  stands 

for  1,  if  i= j ;  0,  if  i^ j . 


10 


n   n 
Q(x)  =  x'Ax  =   V    I   a.  .x.x. 
"  "    i=l  j=l  *?    X  ^ 

is  called  a  quadratic  form  in  x.   The  quadratic  form  x'Ax  and 
its  matrix  A  are  called  positive  definite  if  whenever  x  7*  0  , 
x'Ax  >  0;  positive  semi-definite  whenever  x'Ax    0  for  all 
x  t*  0  and  x'Ax  =  0  for  some  x  ^  0 ;  and  non-negative  whenever 
x'Ax  (or  A)  is  either  positive  definite  or  positive  semi- 
definite.   (Any  non-negative  matrix  A  of  a  quadratic  form 
x'Ax  is  assumed  symmetric  for  mathematical  convenience  and 
does  not  alter  the  value  of  x'Ax  since 

n   n  n   n   (a. ,+a . . ) 

x'ax  =    y     y    a.  .x.x.  =    y     y      l^    12=   x.x. 

-  "     i=i  j=i    «  1  ^     i£i  j£i        2         *  d 

(A+A') 

=  *'  zj~  2 

A+A ' 
where  —~ —  is  symmetric.)   The  rank  of  a  quadratic  form  x'Ax 

equals  the  rank  of  A. 


11 


III.   RANK  OF  QUADRATIC  FORMS 

This  chapter  (1)  defines  linearly  independent  linear 
forms  and  linearly  independent  linear  restrictions  on  linear 
forms  and  (2)  develops  the  theory  of  rank  determination  of 
quadratic  forms. 

A.   LINEAR  FORMS 

Let  X  4    0  be  an  nxl  vector.   If  x  =  (x, ,...,x  )  is  in 

-  '  1 '    '  n 

E  ,  then  the  linear  combination 
n 

n 
L(x)  =  X'x  =   7   X.x. 
"  "    i=l   X  X 

of  components  of  x  with  coefficients  from  X  is  called  a 
linear  form  with  the  associated  vector  X.   Since  X'x  is  a 
scalar,  X'x  =  x'X.   The  square  of  a  linear  form  is  a  quad- 
ratic form  since 

L2(x)  =  (X'x) (X'x)  =  (x'X) (X'x)  =  x'(XX')x. 

2 

Here,  XX'  is  nxn  symmetric.   The  rank  of  L  (x)  =  r(x'XX'x) 

=  r(XX')  =  r(X)  =  1  by  Theorem  2.1  since  X  ^  0.   So  in  gen- 

2 

eral,  XX1  is  not  positive  definite.   However,  since  L  (x)  = 

2    2 
(X'x)  ,  L  (x)  is  always  non-negative. 

If  L.  ,L~ , . . .  ,L,  are  linear  forms  in  E  ,  they  are  said  to 
be  linearly  independent  if  their  associated  vectors 
X,,X?,...,X,  are  linearly  independent  vectors  in  E  ,  i.e., 
the  rank  of  the  matrix  A  =  (X,  ,X?  , .  .  .  ,X,  )  equals  k. 

Theorem  3.1   Q(x)  is  a  non-negative  quadratic  form  in  E 
of  rank  k  if  and  only  if  there  exists  linearly  independent 


12 


linear   forms    L,  ,L2  , . ...  ,L.     such   that 

k        2 
Q(x)    =      I      L. (x)     for  every   xeE    . 
1=1 

Proof:   Suppose  Q(x)  =  x'Ax  is  non-negative  with  rank  k 

and  associated  matrix  A.   By  Theorem  2.2,  there  exists  a 

non-singular  matrix  C  such  that  C'AC  =  (d.6. .) ,  d.e{0,l}, 
3  -  -  —      1  xj     X 

i=l,...,n.   Define  L.:E  -*En  by  L.(x)  =  c!x  where  c!  is  the 
'    '  i   n   1  J      l  -     -l-       -l 

i  row  of  C  ,  i=l,...,k.  Then,  L.  is  a  linear  form  and 
letting  z  =  (L,  (x)  , .  . .  ,L  (x)  )  =  (c,'  x ,  .  .  .  ,c  *  x)  ,  it  follows 
that  z  =  C   x  or  x  =  Cz.   In  that  case, 

n      p     n     p 

x'Ax  =  z'C'ACz  =  z(d.6.  .)z  =   Y   d.zT  =  )      d.LT(x). 

-  --    - -   l  i]  -    .L,       l  l    .L,   l  l  - 

J      1=1         1=1 

But,  by  Theorem  2.2,  k  is  precisely  the  number  of  non-zero 

d. 's.   Deleting  those  d.  such  that  d.  =  0  and  renumbering 

subscripts,  if  necessary,  yields 

k    2 
Q(x)  =  x'Ax  =  I      L. (x) . 

i=l   1 

Since  C    is  non-singular,  the  vectors  c-j  ,-....,  ci  are  linearlv 
independent  and  so  then  are  the  linear  forms  by  definition. 
Conversely,  suppose  L, , . . . ,L,  are  linearly  independent 
linear  forms.   Then,  by  definition,  there  exists  k  linearly 
independent  non-zero  vectors  L ,.. . ,A,  where  X.  is  asso- 
ciated with  L .  ,  i=l ,  . . .  ,k ,  that  is , 

n 
L.  (x)  =   V   X.  .x. 
j=l   l3  x 


13 


for  every  x  =  (x.,.«.,x  )  in  E  .   Letting  A. 
2    -  1      n      n         ^  -l 

i-l,...,k  the  matrix 


<SiV 


./A.   )  , 

-in 


A  = 


-11  -21  '  *  '  -kl 


-12  -22  '  *  *  -k2 


-In  -2n 


X 


kn 


nxk 


necessarily  has  rank  k  because  the  rows  are  linearly  inde- 
pendent, i.e.,  r(A)  =  k. 
Consider 

k    ? 
Q(x)  =  I      LT(x). 

i=l   X  " 

Now 

L2(x)  =L.(x)L.(x)  =  (A!x)'(a!x)  =x'(A.A!)x 
l  -      l  -   l  -      -l     -l-     -   -l-i  - 

hence 

k  k 

Q(x)  =  I      x' (A.A!)x  =  x' (  I      X    X\)x 
i=l     -i  i  "      i=1   i-i 


k 
i=l 


Ail 


A  •  i  A  •  ^  •   .   •  A  ■  ..  A  . 

ll  i2        ll  in 


A  in  A  I  1     A  .  ~ 

i2  ll   i2 


.  .  A  .  ~  A  . 
i2  in 


A  .  A  .  ,  A  .  A  .  — 
in  ll   in  i2 


in 


x 


nxn 
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Q(x)  =  x' 


k    2  k                k 

)  A  .  n  /  A  .  ,  A  .  ~  .   •   •    /    A.  -A 

.  L 1        ll  . L .        ll  l2          .  L ,    ll 

1=1  1=1                  1=1 


k  k 

V   A.  A.,  I      A.  X.n 
L ,       in  ll  .S   in  i2 


1=1 


i=l 


in 


k  k  k 

X.    Xi2Xil  X.    Xi2     •  *  "  X-    Xi2Xin 
i=l        i=l  i=l 


k 

I 
i=l 


in 


x 


nxn 


=  x'Ax  where   A  =   7   A. A!  . 
-  —         -■-.*«-  -i-i 

i=l 

Notice  that  A  is  symmetric.   Thus,  Q(x)  is  a  quadratic  form 
in  x  and  it  must  follow  that  the  rank  of  Q(x)  equals  r(A) . 
Observing  that  A  is,  in  fact,  the  product  of  A  and  A', 

A  =  AA" , 

and,  applying  Theorem  2.1,  r(A)  =  r(AA')  =  r(A)  =  k.   Con- 

2 
sequently,  r(Q(x))  =  r(A)  =  k,  and  since  L.(x)  is  non- 
negative  so  also  then  is 


£   L  (x)  =  Q(x) .   Q.E.D. 
i=l   X 


B.   LINEAR  RESTRICTIONS 

A  vector  a   =    (a,,..., a,)  in  E,  is  called  a  linear  restric- 


tion on  the  set  of  linear  forms,  {L,,...,L,}  if  a  /  0  and 

k 
a'L(x)  =   7   a.L. (x)  =  0 
i=l   x  X  " 

for   every   xeE      where   L(x)    =    (L.  f .- .  .  fL.  )  . 

n  J.  ic 


15 


As  will  be  seen,  a  linear  restriction  has  the  effect  of 

reducing  the  rank  of  a  quadratic  form  by  one.   Another  linear 

restriction  will  further  reduce  the  rank,  only  if  it  and  the 

first  are  linearly  independent.   In  general,  it  is  important 

to  find  the  total  number  of  linearly  independent  linear 

restrictions  on  a  given  set  of  linear  forms,  {L,  , . . .  ,L,  },  in 

order  to  determine  the  rank  of  the  quadratic  form, 

k    . 
Q(x)  =  I      LT  (x)  . 

i=l   1 

Theorem  3.2   If  there  are  exactly  m  linearly  independent 

linear  restrictions  on  the  linear  forms,  L,  , .  . .  ,L,  ,  (m<k) 

then  the  rank  of 

k   2 
Q(x)  =  I      Lf(x)    xeEn 
i=l 

is  k-m. 

Proof:   Suppose  a,,..., a   are  linearly  independent  linear 

restrictions  on  the  linear  forms,  L, ,...,L,  ,  where  a.  = 

(a . ,  , . .  .  ,a . ,  )  for  each  i=l , . . . ,m.   Then 

il'    '  lk  '    ' 


y  a.  -L.  (x)  =   V   a.  .X'.x 

,=i  ^  ^  -    >!  id-:- 


for  all  x  in  E  ,  i=l,...,m,  where  X.    is  associated  with  the 
n  -j 

linear  form  L . (x)  for  each  j=l,...,k.   Recall  that 

k 
Q(x)  =  I      LZ(x)  =  x' (AA')x 
i=l   L 


16 


where 


A'  = 


A-iT   •   •   •   A-. 

11        In 


Akl  •  •  •  Xkn 


*i 


*k 


Let  A  = 


si 


a' 
-m 


then  AA'x  = 


1  =  1  ±3    : 


=  0 


I   a  . A  .x 


for  all  x  in  E  .   Hence  AA '  =  0.   r(A)  =  m  by  hypothesis, 
and  by  Theorem  2.3,  r(A')  <_   k-m. 

Suppose  m  is  the  maximum  number  of  linearly  independent 
linear  restrictions  on  the  linear  forms,  L, ,  ...,L,  ,  and 
r(A')  <  k-m.   Since  r(A)  =  m,  there  exists  an  mxm  sub-matrix 
A*  of  A  such  that  | A* |  ^0.   Without  loss  of  generality,  let 


A*  = 


a 


11 


a 


lm 


a 


ml 


mm 


Since  r(A')  <  k-m,  the  last  k-m  rows  of  A',  X ',...,  A.1  , 

'  '  -m+1 '     -k 

among  others,  are  linearly  dependent.   Then  there  exist 
scalars  b,,...,b,  ,  not  all  zero  (say  b,  ^0)  ,  such  that 


b.  A'  ,  +  ...+  b.  A/  =  0. 
1  -m+1  k  -k 
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Let  8  =  (0,...,0,b   .,...,bj,   Then  certainly, 

k  k  k 

I      3^.(5)  =  I   b.  A!x  =  (  I      b.  A!)x  =  Ox  =  0 
i=l  m+1  m+1 

so   that   a,,...,  a    ,8    are    linearly   independent.      For   suppose 


Then 


c.    ou    +    .     .     .    c      a      =    c    ,  ,  8    =    0 . 
1-1  m   -m  m+1- 


c,  a,  ,  +    .    .     .    +   c   a    ,  =    0 

1    11  m  ml 


cia-i~           +    .     .     .    +  c   a                                           =    0 

1    lm  m  mm 

c,  a,       ,,+...+  c   a         ,,    +    c    L1 B    ,n    =    0 

1    l,m+l  m  m,m+l          m+1   m+1 


c^a,,     +  .  .  .  +  c  a  ,     +  c  .  ,  8,      n 
1  Ik  m  mk       m+1  k   =0 

or  letting  c  =  (c, ,... ,c    )  ,  the  first  m  equations  can  be 

written  A*  c  =  0.   Since  A*  is  non-singular,  £  =  0  ,  that 

is ,  c,  =  .  .  .  =  c   =0.  and  the  last  k-m  equations  become 
1  m  ^ 

cm+l8m+l  -  ° 


cm+lBk   =  °- 

But  8,  ^   0.  hence  c  ,,  =  0.   Consequently,  a,  ,...,a  ,8  are 
k     '        m+1  ^     x  '    -1     -m  - 

linearly  independent  contradicting  the  maximality  of  m. 
Hence  r(A')  =  k-m  and  since  r(A')  =  r(AA')  by  Theorem  2.1 
r(Q(x) )  =  k-m.   Q.E.D. 
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Corollary  3. 3   If  there  is  at  least  ra  linearly  independ- 
ent linear  restrictions  on  the  linear  forms,  L,  ,  ...,L,  ,  (m<k) 

then  the  rank  of 

k 
Q(x)  =  I      hUx) 
ill  x 

for  all  x  in  E   is  less  than  or  equal  to  k-m. 
n    3 

Theorem  3.4   The  linear  forms  L,,...,L,  are  linearly 
independent  if  and  only  if   there  are  no  linear  restrictions 
on  L _  t  m  .  ■  /L-.  • 

Proof:   Suppose  L,  , .  .  .  ,L,  are  linearly  independent  and 
there  exists  at  least  one  linear  restriction,  a  =  (a,  ,...,ou) 
on  L,,...,L,  such  that  a,L,  +  .  .  .  +  a,L,  =  0.   Hence,  one 
of  the  linear  forms,  say  L, ,  can  be  written  as  a  linear  com- 
bination of  the  remaining  k-1  linear  forms 

a2  ak 

Ll  =  "  ^L2  -    '     '     '    ~   3^Lk    (al  *  0) 

=  e2L2  +  '  '  '  +  ekLk 


a  . 
where  $.  =  -  — 1,  j  =  2,...,k. 

Consider  the  vectors,  /U  , ..  •.*]&./    associated  with  the 
linear  forms,  L.,...,L,  ,  and  the  matrix  formed  by  these 
vectors , 


A'  = 


11 


.  X 


In 


A,  -.      .   .   .  A-, 
kl         kn 


x-i 


x-w 
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where  by  hypothesis,  r(A')  =  k.   Now 
Ll  ~    ^2-2-  +  •  •  •  +  ^v-^v* 

k  k 

=  (  7   3. A. , )x,  +  .  .  .  +  (  Y   e.X.  )x 
•  £2   i  il   1  .£2   i  in   n 

which   implies 


.1,  hhi 

i=2 


h  = 


i=2 


e.x. 


—     [  Aj  t  •  •  •  i  A,   J  3 


where  3  =  (3~  ,  .  .  .  ,3^.)  •   Hence,  A,  is  a  linear  combination  of 
the  remaining  k-1  rows  of  A1,  i.e.,  r(A')   :  k-1  which  contra- 
dicts the  linear  independence  of  the  linear  forms  L,,...,L,  . 

Therefore,  there  are  no  linear  restrictions  on  L.  ,...,L.  . 

Ik 

Conversely,  suppose  there  are  no  linear  restrictions  on 

the  linear  forms,  L, ,...,L,  .   Letting  m  =  0  in  Theorem  3.2, 

the  rank  of 

k 
Q(x)  =  I      Lf(x) 

i=l   1 

2 

for  all  x  in  E   equals  k-m  =  k.   Since  L. (x)  is  non-negative, 

so  then  is  Q(x).   Upon  applying  Theorem  3.1  the  linear  forms, 
L,  ,...,L,  ,  are  seen  to  be  linear  independent.   Q.E.D. 
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IV.   APPLICATIONS  TO  ANALYSIS  OF  VARIANCE 

Knowledge  of  the  rank  of  a  quadratic  form  is  essential 
in  testing  for  the  equality  of  means  of  k  normal  populations 
having  the  same  variance.   This  statistical  method  is  called 
the  analysis  of  variance.   This  chapter  demonstrates  the  use 
of  linearly  independent  linear  restrictions  of  linear  forms 
to  determine  the  rank  of  quadratic  forms. 

A.   ONE-WAY  CLASSIFICATION 

Suppose  that  there  exists  k  groups  of  independent  obser- 
vations 

Yll"  '  '  ,ylnn  ,y21"  "  'y2n  '•  "'Ykl'-"*  ,ykni 
12  k 

from  normally  distributed  populations  with  means  u,,...,u, 

2 

all  with  the  same  variance  a    .   Thus  the  model  is 

v..  =  u .  +  e .    i= 1 , .  .  .  ,  k   -i-l,...,n. 

Ji]     1     ij      '    '    J   '    '  l 

where  y .  are  fixed  constants  and  the  e .   are  independent 
l  ij  r 

2 

random  normal  deviates  with  zero  mean  and  variance  a  . 

Let  x.   =  —  (y.  .-ii.)  ,   Consider  the  identity 

1  j    a  -*  i  j  '  i  J 


x.       =  (x.  .-x.  )  +  (x .  -x)  +  X  (1) 

ID      ID   i       i 


where 


n . 

l 


x.  =  —    }   x .  .    1=1 , . . .  .k 

i    n.  >.   i]  ' 

l  j  =  l         J 


1   k   - 

x  =  —   y    x . . 

n  .  S   i 

i=l 


:j 


Squaring  both  sides  of  equation  (1)  and  summing  over  i  and  j , 
i=l,...,k,  j=l,...,n.  yields  the  identity 


I         I      x..  =  I         I       (x..-x.)   +  I      n  (x  -x)   +  nx 
i=l  j=l  13         i=l  j=l    1:  X  i=l   K   1 


Q  =  Q1   +   Q2  +  Q3. 


Since  y.  .  is  N(u.  ,0    )  then  x.   is  N(0.1)  and 

k   1 


Q-  .1   .1  JtJ 


is  x  (n)  an<^  nas  rank 


i=l  j=l 


n  =   V   n. 
i=l 


The  ranks  of  Q,  ,  Q2  ,  and  Q.,,  in  turn,  can  be  found  by  find- 
ing the  number  of  linearly  independent  linear  restrictions 
associated  with  each  quadratic  form  and  applying  Corollary 
3.3  and  Theorem  2.4. 
Consider  first 


1   n. 
k    1 


-  x  2 


i=l  i=l    J 


where  x  =  (x,.,...,,x.   ' *  * ' 'xkl ' *  *  * 'xkn  ^'      Let  Lii  ^  = 

(x.  .-x.)  =  X!  .x  be  a  linear  form  in  x  where  X.  .  is  an  nxl 
iD   1     "ID"  "       "ID 

vector 


n . 
1 


i=l 

y   n   rows 
u=l 
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A.   = 


l 

n . 

1 

i  -  i 

n . 

i 


n . 

l 


1 

n 


i=l 


(  I      n  +j)    row 


u=l 


u 


i=l 
-— (  I      nu) 
u=l 


th 


row 


i=l 
n-  (  y   n  )  rows 
u=l 


nxl 


For  each  i=l.,,,.k  there  exists  an  nxl  vector  a,  such  that 


n 


a!L(x)  =    ^  aijL..(x) 


j  =  l 


n . 

l 


y    (x. .-x.)  =  o 


3  =  1 


where 


-1  = 


1 
0" 


i=l 

y    n   rows 
u=l 


i=l 


th 


(  )    n  )    row 

u=l 


nxl 
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and 


L(x)  = 


Xll  Xl 


X-   -x. 

In,   1 


xkl  xk 


x,   -x, 

kn,   k 
k 


nxl 


Forming  the  matrix  A  =  (a,,..., a,) 

1  0  ...  0 


A  = 


n,  rows 


n2  rows 


n,  rows 
nxk 


it  can  be  seen  that  the  columns  of  A  are  linearly  independent 
i.e.,  r(A)  =  k;  hence  by  definition,  the  set  of  k  vectors 
{a,,...,a,  }  are  linearly  independent  linear  restrictions  on 
the  linear  forms,  L. ,  , . . .  ,L,   , . . . ,L,  ,  , . . .  ,L,   .   Applying 

Corollary  3.3,  r(Q,)  _<  n-k. 
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In  a  similar  manner  for 


-   -  2 
Q2(x)  =  I      ni(xi-x) 

i=l 


where  x  =  (x,  , .  .  .  ,x,  )  let 


Li(x)  =  /n7  (xi-x)   (i=l,...,k) 


=  X!x 
-l- 


be  a  linear  form  in  x  where  X .  is  a  kxl  vector 

-l 


/57  (-±) 


X.   = 

-1 


/H-  (-|) 


l      k 


*I  ("? 


L 


n.  (-  £■) 
i     k 


.th 
i    row 


kxl 


Likewise,  there  exists  one  kxl  vector  a  such  that 


a'L(x)  =   y   a.L.  x 
-  -  -     .  L  -.       11  - 
i=l 


I   a.  {/n7(x.  -x)  } 
. L ,   i    i   i 
i=l 


=   T   /n7(x.-x) 
i=l    x   x 

=  0 
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where 


a  = 


and  L(x)  = 


kxl 


/n^(x1-x) 


v/R^(xk-x) 


kxl 


Here  a  represents  at  least  one  linear  restriction  on  the 
linear  forms,  L.  (x)  , . .  .  ,L,  (x)  ,  and  by  Corollary  3.3 


r(Q2)  <   k-1. 


-2  . 


Lastly,  Q3(x)  =  nx   is  the  square  of  a  single  linear 
form,  L(x)  =  /nx,  and  has  rank,  r(Q3)  =  1. 


In  summary, 


r(Q)  =  n 

rfC^)  <_  n-k 

r(Q2)  <  k-1 

r(Q3)  =  1 


where  Q  =  Q,  +  Q   +  Q3.   Hence 

r(Qx)  +  r(Q2)  +  r(Q3)  _<  n  -  k  +  k 
but  by  Theorem  2.4 


1  +  1  =  n 


r(Q)  =  n  <   r(Q1)  +  r(Q2)  +  r(Q3). 


Consequently , 


and 


r(Q1)  +  r(Q2)  +  r(Q3)  =  r(Q) 


r(Q1)  =  n-k 


r(Q2)  =  k-1 
r(Q3)  =  1   . 

From  Cochran's  Theorem  the  quadratic  forms  0, ,  Q2 ,  and  Q3 

2 
are  linearly  independent  and  are  x   distributed  with  n-k, 

k-1,  and  1  degrees  of  freedom,  respectively. 
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The  foregoing  procedure  for  finding  at  least  a  lower 

bound  for  the  rank  of  a  quadratic  form 

k 
Q(x)  =  I      L?(x) 
i=l   1 

endeavors  to  construct  a  matrix  of  known  linear  restrictions, 

A  =  [o^,.  .  .  ,§.  ] 

whose  rank  is  necessarily  the  number  of  linearly  independent 
linear  restrictions  on  the  linear  forms  L,  ,  ...,L,  .   It  could 
be  shown  that  if  the  linear  forms  L, , . . . ,L,  are  linearly 
independent  then  Theorem  3.1  can  be  applied  to  give  the  rank 
exactly.   Such  a  procedure  entails  finding  the  rank  of  the 
matrix  of  coefficient  vectors,  A  =  [Xw...,X,  ]  associated 
with  the  linear  forms  L,  , .  .  .  ,L,  .   In  general,  the  determina- 
tion of  the  rank  of  A  is  complicated  by  the  odd  construction 

of  the  X . ' s .   On  the  other  hand  the  matrix  of  the  known  lin- 

-l 

ear  restrictions,  A,  contains  only  elements  equal  to  zeros 
or  ones.   Hence,  the  easier  method  of  finding  information 
concerning  the  rank  of  a  quadratic  form  is  to  look  for  the 
linear  restrictions  and  apply  Theorem  2.2  or  Corollary  3.3. 
In  the  next  section  two-way  classification  of  analysis 
of  variance  is  investigated  and  it  will  be  seen  that  although 
the  matrix  of  known  linear  restrictions  becomes  larger  deter- 
mining its  rank  remains  relatively  easy. 
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B.   TWO -CLASSIFICATION 

In  the  one-way  classification  of  the  analysis  of  variance 
k  groups  of  independent  normally  distributed  observations  are 
recorded.   The  k  groups  represent  k  different  variations  of  a 
particular  variable  or  factor  which  is  needed  to  yield  a  de- 
sired result,  e.g.,  the  observation  itself.   It  is  the  purpose 
of  the  analysis  of  variance  to  ascertain  if  there  is  any  sig- 
nificant differences  in  the  results  of  any  of  these  variations. 
The  measurement  of  crop  yield  using  different  types  of  ferti- 
lizer is  an  example  of  such  a  grouping  or  classification. 

In  a  two-way  classification  of  the  analysis  of  variance 
the  results  of  an  experiment  are  classified  according  to  var- 
iations of  two  influencing  variables.   In  the  crop  growth  ex- 
ample, crop  yield  can  be  classified,  not  only  by  different 
types  of  fertilizer,  but  also  by  various  soil  compositions. 

Suppose  there  exists  r  variations  of  one  classification 
and  c  variations  of  a  second  classification.   Consider  n 
observations  taken  from  each  of  re  possible  combinations  of 
the  two  classifications.   In  tabular  form 


*11 


*1< 


Yri  •  •  •  Y 


re 
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where  y  =  (y..,,...,y.   )  is  an  nxl  vector.   It  is  assumed 
•*■     -*  13I '     2  ljn 

that  the  re  sets  of  n  observations  are  random  samples  from 

re  separate  populations,  each  normally  distributed  about 

2 

mean  y .   but  all  with  the  same  variance  a  .   The  model  is 

i: 

y .  .      =   y .      +    e .  i=l , . . .  ,r 

v=l , . . .  ,n 

2 
where    e .  .      is   N (0  ,a    ) .      Let 
ljv 

y .  .    -y  .  . 

2 1JV       1] 

x.  .      =   — * *■ 

ljv  a 

and  consider  the  identity 

x.  .   =  (x.  .  -x.  .)  +  (x.  .-x.  -x  .+x)  +  (x.  -x) 
13V     13V   13       lj   i*   »j         i« 


(2) 


+  (x  .-x)  +  X 

'1 


where 

1   n 

x.  .  =  —   y    x.  . 

in    n  L ,      liv 
J     v=l    J 


c   n 


x.    =  —   y     y    x. .    =—   y  x. . 

i*    en  Z,  L .       liv   c  . L ,    in 
3=1  v=l    J  3=1   J 


1    r   n  r 

x     =  —   y     y    x. .    =  —   y    x. . 

•  j    rn.fi,  L ,   13V   r-ii   13 


i=l  v=l    J      i=l 


n  r  c 


x     =  —   y     y     y    x. .    =—   j  x.    =—   y 

rcn  .  L,  .L.      L~      liv   r  .*•,  i"    c    ,L~       »i 
i=l  3=1  v=l    J      i=l        3=1    J 

Squaring  both  sides  of  equation  (2)  and  summing  over  i,  j  and 

v,  i=l,...,r,  j=l,...,c,  v=l,...,n  yields  the  identity 


29 


r    c   n 


r   c 

J 


r   c   n 

I        I        I 
i=l  j=l  v=l 

-s  2 


(x. .  -x. .)2 

1=1  J=l    v=l     J 


+  n   y    V   (x. .-x.  -x  ,+x)   +  nc   y   (x 

•  -i  •  i    ID   i"   *D  •  i    ■ 

i=l  -|=1     J        J  i=l 


-V2 


-x) 


v         -         -    2  -2 

+  nr   g   (x  -~x)   +  rcnx 
j  =  l    •? 

Q  =  Qx  +  Q2  +  Q3  +  Q4  +  Q5  . 

2 

The  rank,  of  Q  is  rcn  since  it  is  x   with  rcn  degrees  of 

freedom.   The  ranks  of  Q. ,...,Q-  can  be  determined  by  finding 
the  number  of  linearly  independent  linear  restrictions  asso- 
ciated with  each  quadratic  form  and  applying  Corollary  3.3 
and  Theorem  2.4. 

Following  the  procedure  of  the  previous  section  consider 
first 


Ql(5>  -  I        I        I      (xiiv"^ii 
x      i=l  i=l  v=l   13V  1D 


where 


x  = 


Sill 


-lc 


-rl 


with  x . . 
"ID 


x 

L'rc. 


rcnxl 


;iji 


X.  . 

ljn 


L 


X.  . 

i:n 


i=l , 

D  =  l, 


.  ,r 
•  ,c 


nxl 


nxl 
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Let  L.   (x)  =  (x. .  -x. .)    i=l.. 
1]V  -        1]V   in      .  , 


=  X!  .  x 
-ijv- 


v=l , . 


. ,r  be  a  linear  form  in  x 
.  ,c 

.  ,n 


where  X . .   is  a  rcnxl  vector 

-lJV 


X. 
-13V 


1  - 


1 
n 


1 

n 

1 

n 

1 

n 


1 

n 


{  (i-1) c+j }n  rows 


th 


{ (i-1) c+j }n+v    row 


rcn-{ (i-1) c+j+l}n  rows 


0    y   rcnxl 
J 


For  each  i,  j,  i=l,...,r,  j=l,...,c  there  exists  an  rcnxl 


vector  a. .  such  that 
13 


n 


a!  .L(x)  =   7   a.   L.   (x) 
-iD   "     yii      idv  idv  - 


n 


=  I       (x.  .  -x.  .) 

L,  1]V   11 

v=l     J     J 


=  0 
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where 


X.  .  = 
-ID 


(i-l)c+j  n  rows 


rcn- (i-1) c+j+1  n  rows 


rcnxl 


and  L(x)  =  (Li jv(x) ) rcnxl /  i=l,...,r,  j=l,...,c,  v=l,...,n. 

Note  that  there  are  re  such  vectors,  a. ..   Forming  the  matrix 

-ID 


using  the  nxl  sub-matrices 

1 


1  = 


and   0  = 


nxl 


nxl 


A  becomes 
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A  = 


1  0 
0  1 


0  0 


0  0  0 
0  0  0 


10  0 


0  0 
0  0 


0  0  0 
0  0  0 


0  0 

0  0 


0  0 


1  0 
0  1 


00...000...0...  00...1 


rcnxrc 


It  can  be  seen  that  the  columns  of  A  are  linearly  independent, 
i.e.,  r(A)  =  re;  hence  by  definition,  the  set  of  re  vectors, 
{a..;  i=l,...,r,  j=l,...,c}  are  linearly  independent  linear 
restrictions  on  the  linear  forms  {L. .  (x) ;  i=l,...,r, 

1}V  -  lit 

j=l,...,c,  v=l,...,n}.   Applying  Corollary  3.3, 
r(Q,)  <_   rcn-rc  =  rc(n-l). 

In  a  similar  manner  for 


Q0  (x)    =   n      7         I       (x- •~x-    ~x    -+x) 


x) 

WnGJTG     X     —       *      l  i   t  ■  •  *   /      i       /  •  •  •   f "     i    /  •  *  •  /  **•         /       JL6"C 


i-i  j-l  x-      '^ 


'li 


■lc 


'rl 


re 


L..(x)    =    (x..-x.    -x    .+x)         i=l,...,r 
ID    "  ID       i*       'D  i.1  ' 

J  — X,.    .    .    ,  t- 

=    X!  .x 
-iD- 
be    a   linear    form   in   x  where    X  .  .    is    a   rcxl   vector 

-ID 
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X.   = 


_i 

re 


JL 

re 

r    re 
__1 
re 


_1 
re 


1  - 


1 

+ 

1 

c 

• 
• 
• 

re 

1 

+ 

1 

c 

re 

1 

i  + 

c 

1 

r 

re 

1 

+ 

J^ 

c 

• 

re 

1 

+ 

1 

c 

re 

1 

re 

_1 

re 

i'+  -i 

r   re 
JL 

re 


_1 
re 


.th 
■j         row 


(i-l)c    row 

/  •  n  \   • th 
(i-l)c+3    row 

.  th 
■lc    row 
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__1 

re 


_1 

re 

i    -i 

-  +  re 

_1 
re 


_1 
re 


/   t \  ■ • th 
(r-l)c+j    row 


rcxl 


Likewise  there  exist  r+c  vectors  {a . ;i=l , . . . ,r}  and 


-l 


(3.;  j=l,...,c}  such  that 


a!L(x)  =   V   a. .L. . (x] 
i=l   ^  ^  " 


y    (x. .-x.  -x  .+x) 

iil   ^  1#  '1 


3 

=  0 


and 


g'L<x)  =     i     BijL     (x: 


J. 

=   y    (X. .-X .  -X  .+ 
i=l     l3    !'    '1 


X) 


=  0 
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where 


r  oi 


c  rows 


c  rows 


2l  = 


c  rows 


ith  block 
of  c  rows 


c  rows 


c  rows 
'rcxl 


c  rows 


respectively , 
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Forming   the   matrix 
A  = 


[o^,  .  . 


*    '^v-'—l    *  "    *    *    '  —CJ 


1 

0 

1 

0 

•     • 

1 

0 

•    • 

0 

1 

•    • 

0 

1 

•  • 

0 

1 

•    • 

0 

0 

0 

0 

•  • 

0 

0 

.  . 

0 

1 

0 

0 

• 

0 

0 

1 

• 

0 

• 

0 

0 

0 

•       4 

1 

, 

0 

1 

0 

• 

, 

0 

• 

0 

0 

1 

•     « 

i    m 

0 

• 

0 

0 

0 

•       4 

1 

■ 

1 

1 

0 

1     • 

0 

• 

1 

0 

1 

•  • 

'  • 

0 

• 

1 

0 

0 

. 

,  . 

1 

rcx(r+c) 


it    can   be   seen  that 


I      «.    =      I      3.     , 
i=l  j=l      J 

hence  the  rank  of  A  is  at  most  r+c-1.   Consider  the  (r+c-1) 

x(r+c-l)  sub-matrix  of  A, 


A*    = 


10    0 
10    0 


0    0    0. 
0    0    0. 


0    0    10    0 
0    0    0    10 


1 

0 

0    .  , 

,  .    0 

0 

0 

0 

0 

1 

0 

0    .  . 

.    0 

0 

0 

0 

0 

0 

1 

0    .  , 

.    0 

0 

1 

0 

0 

0 

0 

1  . . 

.    0 

0 

0 

1 

0 

.10    0    0    0. 
.01000. 


c   rows 


r-1    rows 
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Now  | A* |  =  +  1  depending  if  r+c-1  is  odd  or  even.   Consequent- 
ly, r(A)  =  r+c-1  and  by  definition,  the  set  of  vector 

{a,,..., a  ,  $..,...,$  )  represent  r+c-1  linearlv  independent 
—l      — r   -l      -c  ..  i. 

linear  restrictions  on  the  linear  forms  {L. .  (x)  ;  i=l,...,r, 
j=l,...,c}  and  by  Corollary  3.3,  r(0~)  <_   rcn-r-c+1  = 
(r-1) (c-1) . 

Consider  next 

r    -    -  2 
Q^(x.)  =  nc  I       (x. #-x) 

1=1 


where  x.  =  (x.  ,...,x   ).   Let 
1*       r* 

Li.  (*-)  =  U.  ;rx) 

=  X!  x. 

-l  •  - 


i=l , . . . ,r 


be  a  linear  form  in  x.  where  X.   is  a  rxl  vector 

-l . 

l" 
r 


1 

r 


i  -i 

r 

1 
r 


1 
r 


.th 
■l    row 


rxl 


There  exists  only  one  vector  a  such  that 


a'L  x.   =   T   a .L .  (x. ) 
---      .  *<,   li--. 
1=1 


=  I       (x   -x) 
i=l    x 


=  0 
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where 


a  = 


rxl 


Hence  a  represents  one  linear  restriction  on  the  linear  forms 
{L.  (x.);  i=l,...,r}  and  by  Corollary  3.3,  r(Q-J    r-1. 

By  an  analogous  argument  r(Q.)  _<  c-1. 

-2 

Lastly,  Qr(x)  =  rcnx   is  the  square  of  the  single  linear 


form,  L(x)  =  /rcn  x,  and  has  rank,  r(Q5)  =  1 


In  summary, 


r(Q)   =  rcn 


r(Q- 
r(Q. 
r(Q. 
r(Q, 
r(Q, 


<_   rcn-rc 
<_  rc-r-c+1 

<  r-1 

<  c-1 
=  1 


where 


Hence , 


Q  =  Q1  +  Q2  +  Q3  +  Q4  +  Q5. 


[   Q,  <  rcn-rc+rc-r-c+l+c-l+r-1+1  =  rcn 
k=l   K  " 

but  by  Theorem  2.4 

5 
r(Q)  =  rcn     £   r(Q,  )  . 
k=l 

Consequently , 

r(Qx)  +  r(Q2)  +  r(Q3)  +  r(Q4)  +  r(Q5)  =  r(Q) 
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and 

r(Q1)  =  rc(n-l) 

r(Q2)  =  (r-1) (c-1) 

r(Q3)  =  r-1 

r(Q4)  =  c-1 

r(Q5)  =  1. 

From  Cochran's  Theorem  the  quadratic  forms  Q-w...,Qr 

2 
are  linearly  independent  and  are  x   distributed  with  degrees 

of  freedom  equal  to  their  respective  ranks. 
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V.   CONCLUSION 

In  general  it  may  be  difficult  to  verify  that  the  linear 

forms  L,  (x)  , . . .  ,L,  (x)  are  linearly  independent  directly,  or 

when  they  are  not,  to  find  the  maximum  number  of  linearly 

independent  linear  restrictions  on  them  in  order  to  determine 

the  rank  of  the  associated  quadratic  form 

k 
Q(x)  =   |   L.  (X)  . 
i=l   1 

Fortunately,  it  is  sufficient  only  to  observe  that  if  there 
are  at  least  m  linearly  independent  linear  restrictions, 
then  the  rank  must  be  less  than  or  equal  to  k-m.   Enough 
information  is  often  then  available  to  establish  that  no 
more  linear  restrictions  exist.   Hence,  indirectly,  the  rank 
is  found  to  be  exactly  k-m.   The  application  of  rank  deter- 
mination to  analysis  of  variance  is  an  example  of  this  in- 
direct procedure. 

It  may  now  be  stated  that  the  procedure  as  outlined  by 
Cramer  [1]  and  Hald  [2]  for  determining  the  rank  of  quad- 
ratic forms  has  been  explicitly  justified  in  detail  and 
illustrated. 
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